building signal-android in WSL2

2021-12-15 · 8 min read

Be warned, these instructions are more devlog, less polished tutorial.

Build #

swtich from non-existant motherboard-connected TPM to integrated Intel CPU TPM + restart > BIOS > Advanced\PCH > TPM -> Firmware TPM
Windows PC Health Check says machine meets all requirements, but windows update incorrectly refuses to upgrade + Manually run Windows11 Installation Assistant (https://www.microsoft.com/software-download/windows11)
Windows11 Installation Assistant errors with cryptic error code 0x8007007f + Need to explicitly run as administrator
Update Nvidia GeForce drivers to 510 (https://developer.nvidia.com/cuda/wsl)

Update wsl

# In Powershell/cmd (run as admin)
$ wsl --update
$ wsl --shutdown

test GUI apps (in WSL)
```
$ sudo apt-get install x11-apps
$ xcalc
```
This should run an x11 calculator GUI app in windows.

Install android sdks (https://developer.android.com/studio) + Download android studio and extract into ~/android-studio.

Probably copy these snippets that I have in my ~/.bashrc

# file: ~/.bashrc

# ..

# Java SDKMAN
export SDKMAN_DIR="$HOME/.sdkman"

# Android
export ANDROID_SDK_ROOT=$HOME/android
ANDROID_STUDIO=$HOME/android-studio
ANDROID_SDK_VERSION=31.0.0
ANDROID_PATH=$ANDROID_SDK_ROOT/cmdline-tools/latest/bin
ANDROID_PATH=$ANDROID_PATH:$ANDROID_SDK_ROOT/build-tools/$ANDROID_SDK_VERSION
ANDROID_PATH=$ANDROID_PATH:$ANDROID_SDK_ROOT/tools
ANDROID_PATH=$ANDROID_PATH:$ANDROID_SDK_ROOT/tools/bin
ANDROID_PATH=$ANDROID_PATH:$ANDROID_SDK_ROOT/platform-tools

# ..

# PATH
export PATH=$PATH:$ANDROID_PATH

# ..

# Java SDKMAN
[[ -s "$HOME/.sdkman/bin/sdkman-init.sh" ]] && source "$HOME/.sdkman/bin/sdkman-init.sh"

Install some missing x11 lib needed to run android studio.
```
$ sudo apt-get install libxtst6
```

Install JDK 1.8 (version required by Signal) and JDK 11 (version required by gradle)

$ curl --proto '=https' --tlsv1.2 -sSf "https://get.sdkman.io?rcupdate=false" | bash
$ source ~/.bashrc
$ sdk install java 8.0.302-open
$ sdk install java 11.0.12-open

Open android studio
```
$ ~/android-studio/bin/studio.sh
```
Ensure you choose the newly installed JDK1.8 when following initial setup.

Download generic SDK, SDK v31, and device emulator into $HOME/android directory. My bashrc should already have $PATHs setup for this.
Update android studio plugins. Try building a sample app (I chose Sunflower)

Install additional SDK libs

$ sdkmanager --sdk_root="${ANDROID_HOME}" --update
$ sdkmanager --sdk_root="${ANDROID_HOME}" --install \
	"cmdline-tools;latest" \
	"extras;google;m2repository" \
	"extras;android;m2repository" \
	"extras;google;google_play_services"

Try building Signal-Android

$ git clone ..
$ cd Signal-Android
$ ./gradlew clean assemblePlayProdRelease

$ cd app/build/outputs/apk/playProd/release/
$ ls
Signal-Android-play-prod-arm64-v8a-release-unsigned-5.27.13.apk    Signal-Android-play-prod-x86-release-unsigned-5.27.13.apk
Signal-Android-play-prod-armeabi-v7a-release-unsigned-5.27.13.apk  Signal-Android-play-prod-x86_64-release-unsigned-5.27.13.apk
Signal-Android-play-prod-universal-release-unsigned-5.27.13.apk    output-metadata.json
# Sign apk so we can install
# Since I already built/installed a sample app, there's already debug
# keys generated from android studio in `~/.android/debug.keystore`.
# Note: the default password is "android"
# EDIT: doesn't work $ jarsigner -verbose -keystore ~/.android/debug.keystore Signal-Android-play-prod-x86_64-release-unsigned-5.27.13.apk androiddebugkey
$ apksigner sign \
	--verbose \
	--v4-signing-enabled \
	--ks ~/.android/debug.keystore \
	--ks-pass pass:android \
	--in Signal-Android-play-prod-x86_64-release-unsigned-5.27.13.apk \
	--out Signal-Android-play-prod-x86_64-release-signed-5.27.13.apk

Setting up virtual device

$ sudo adduser $USER kvm

# in WSL2 need to do this every restart...
$ sudo chgrp kvm /dev/kvm && sudo chmod g+rw /dev/kvm

# verify
$ ls -la /dev/kvm
crw------- 1 root kvm 10, 232 Dec  8 11:46 /dev/kvm

In the IDE AVD Manager, choose an image to download. IDK what is good lmao

+If avdmanager list returns Exception .. NoClassDefFoundError .. XmlSchema + Default platform-tools fail when run with java 11 but not java 8 + Works with the cmdline-tools/latest versions. Those are now set higher prio in my $PATH

Installing random missing dependencies

$ sudo apt-get install libnss3 libxcomposite1

can verify kvm should work

$ sudo apt-get install cpu-checker
$ kvm-ok
INFO: /dev/kvm exists
KVM acceleration can be used

Testing emulator on cli

# Why does it fail when run outside of this directory?????
$ cd ~/android/emulator
$ ./emulator -avd Pixel_2_API_31

Holy shit it actually worked lmao. It complains a bit about "nested emulation" not supported well.

It's not clear that this made any sort of difference?

# file: /mnt/c/Users/phlip9/.wslconfig
# Settings apply across all Linux distros running on WSL 2
[wsl2]
# Enable nested virtualization
# I believe this will improve perf when running an Android emulator inside WSL2?
nestedVirtualization=true

Reporting error connecting to adb daemon on some port 5XXX. Dunno if this is important or not.

Successfully ran "Sunflower" sample app on emulator!! :D

Testing Signal-Android on emulator

Check that device is visible from adb and we can connect

$ adb devices
List of devices attached
emulator-5554	device

$ adb -s emulator-5554 shell
emulator64_x86_64_arm64:/ $ uname -a
Linux localhost 5.10.43-android12-9-00031-g02d62d5cece1-ab7792588 #1 SMP PREEMPT Mon Oct 4 21:48:11 UTC 2021 x86_64

$ adb -s emulator-5554 install -r Signal-Android-play-prod-x86_64-release-signed-5.27.13.apk

Run the Signal app

It works!! Need working phone SIM I think to get past setup screen...

Setup google voice (https://voice.google.com) number and use it to get past setup screen
Android Studio / IDEA setup for Signal Android
```
$ cd Signal-Android
# generates .idea project files
$ ./gradlew idea
```
Then start Android Studio and open ~/dev/Signal-Android project directory.

Hit > Load Gradle Project and then > Trust Project

Gradle will complain about some partially supported distributionSha256Sum option in gradle-wrapper.properties. Tell it to STFU and keep going (first option).

Everything should finish first build successfully.
Understanding Signal build variants

In the base build.gradle you can see they define a bunch of different build variants, which end up generating a billion different gradle tasks and really complicating my life.

TLDR; target PlayStagingDebug

AFAICT, there's effectively { Play, Website, Nightly, Study } x { Prod, Staging } x { Debug, Release, Flipper, Perf, Mock } targets +/- some cases.

For developing, I think we want Debug, which uses the debug.keystore for signing (vs actual keys for pushing to Play store).

I'm not sure what Study is, maybe a mock UI or something. Probably target Play or Website.

It looks like Flipper refers to https://fbflipper.com/ which is some nice app UI/UX debug/development thing. IDK how to use it yet tho.

During development, we probably target Staging so don't accidentally fuck up Signal's prod services. You can also see the MobileCoin config switches to testnet on Staging vs Prod. Need to see if hitting Staging works with a real phone #.

Building PlayStagingDebug and then installing onto device/emulator:

In CLI:
```
./gradlew installPlayStagingDebug
```
In IDE:

(gradle tasks must be synced) Gradle Tab > Signal > Tasks > install > installPlayStagingDebug
Understanding Android x Gradle
- When you make changes to any build.gradle or w/e you need to re-sync Gradle in the IDE.
- I also enabled "update Gradle tasks during sync option" Default project structure:
Signal-Android/app/src/test: local unit tests. these are actually run on your local machine and must have no dependency on the Android framework (or only mocks).

Signal-Android/app/src/androidTest: on-device instrumented tests. these tests must run on a device or emulator. they also have an app Context and access to the Instrumentation API for manipulating UIs etc...

I'm having a lot of trouble building + deploying to the emulator via the IDE. Dunno if this is some local config/setup issue but for now going to try just doing all build+deploy from the cli.

Run local unit tests
```
$ ./gradlew testPlayStagingDebugUnitTest
```
Run specific test (more filter rules: https://docs.gradle.org/current/userguide/java_testing.html#test_filtering)
```
$ ./gradlew testPlayStagingDebugUnitTest --tests *.myTestMethod
```
Continuously run tests (or any task)
```
$ ./gradlew --continuous testPlayStagingDebugUnitTest
```

Actually running the unit tests

$ ./gradlew testPlayStagingDebugUnitTest

Unfortunately I got like 33 errors initially, claiming the following:

org.thoughtcrime.securesms.database.model.GroupsV2UpdateMessageProducerTest > you_requested_to_join_the_group FAILED

		java.lang.UnsatisfiedLinkError:
			/tmp/resource8597950725831998403.so: /lib/x86_64-linux-gnu/libm.so.6: version 'GLIBC_2.29' not found (required by /tmp/resource8597950725831998403.so)

This means the installed libc version is too outdated compared to the compiled library's requirements. In this case, I needed to update Ubuntu from a fairly old version (18.04) to the next LTS (20.04).

# Confirm that we're on ubuntu 18.04 right now
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.6 LTS
Release:        18.04
Codename:       bionic

# For some reason need to do this o/w upgrade silently aborts
$ sudo apt purge snapd

# Upgrade and clean up everything before upgrading
$ sudo apt update
$ sudo apt list --upgradable
$ sudo apt upgrade
$ sudo apt --purge autoremove

# Begin the upgrade
$ sudo apt install update-manager-core
$ sudo do-release-upgrade

# also needed to rebuild tmux after this????

After finishing that, all local unit tests completed successfully :^)

$ ./gradlew testPlayStagingDebugUnitTest
# ..
BUILD SUCCESSFUL in 2m 20s
194 actionable tasks: 186 executed, 8 up-to-date

Deploying to real device

First setup ADB over TCP inside WSL2, then simply
```
$ ./gradlew installPlayStagingDebug
```

TODO #

look into "Configure hardware acceleration for the Android Emulator" (https://developer.android.com/studio/run/emulator-acceleration)